FGB RDM Assessment 2023: Results
Jessica Hrudey
2024-11-19
Executive Summary
- Almost all of FGB research involves data from human research
subjects
- All FGB researchers need some level of GDPR awareness, particularly knowledge of when the GDPR does and does not apply to their research.
- Data management practices appear slightly improved since 2019,
particularly data archiving, but there remain issues that continue to
arise despite numerous awareness campaigns within the faculty, such as:
- Inappropriate collection of BSNs; BSNs may not be used for research
- Ongoing usage of non-VU storage/archiving solutions
- Lack of READMEs for archived data
- Lack of PURE registration for archived datasets
- FGB researchers clearly want to participate in data sharing
practices, particularly sharing restricted access data upon request, but
many lack plans for how to carry this out.
- The Research and Policy Support team will use these assessment results to build a new system that better supports data sharing.
Background information
In the latter half of 2023, FGB researchers and research support staff were asked to complete a questionnaire on their RDM practices. A total of 153 FGB staff members responded to the survey: 127 researchers responded with a 70.9% completion rate and 26 research supporters responded with a 84.6% completion rate.
Research supporters were not asked all of the questions that researchers were asked. The questions that were posed to both groups can be identified in this report as underlined header text.
Some questions allowed for multiple answer options to be selected. For the figures representing the results of these questions, the percentages shown do not add up to 100%. This is because the percentages shown are the proportion of people who selected that answer choice from the total number of respondents and there will be some overlap when respondents select multiple answer choices for a given question. Questions that allowed for multiple answers to be selected can be identified in this report by the presence of ** at the end of the header text.
Department Distribution
Which department does the respondent primarily work in?
Research Role
What is the respondent’s primary research function?
Data Management Planning Practices
Questions asked:
- Has a DMP been written for the research project
- Was the DMP written using the VU’s DMPonline
- Has the data management plan been reviewed by a data steward
- Have research support resources been used in the planning of the research project
- Has the research protocol been pre-registered and if so, where is it pre-registered
Has a DMP been written for the research project1?
↩︎Overall response
By department
By research role
Was the DMP written using the VU’s DMPonline?
↩︎Overall response
By department
By research role
Has the data management plan been reviewed by a data steward?
↩︎Overall response
By department
By research role
Have research support resources been used in the planning of the research project?
↩︎
Use of resources
Overall response
By department
General resources used**
Overall response
By department
FGB-specific resources used**
Overall response
By department
Has the research protocol been pre-registered?
↩︎Overall response
By department
By research role
Where has the research protocol been pre-registered?**
Conclusions/Recommendations on DMP Practices
↩︎- DMPs are being written at a comparable rate to the previous
assessment in 2019
- Some confusion may exist for researchers in the Biological Psychology department and the Cognitive Psychology section because their dept/section often uses a generalized DMP for all research
- Uptake of DMPonline appears to be quite good, particularly amongst
PhDs
- To be expected due to required DMP course for 1st-year PhDs
- Use of research support resources is moderately good, particularly
of FGB-specific webpages, but usage of specific support topics should be
improved
- Specific support topics (privacy, security & archiving) are underutilized particularly by ETP and BW research staff
- Pre-registration is not a standard practice for many researchers,
except for those in KNOP
- Awareness and reasoning for pre-registration will need to be increased amongst researchers
Types of Data and Research Populations
Questions asked:
- Is the data used in the research project collected from and/or about human beings
- What research populations are studied in the research project
- What types of data are used in the research project
- What software programs are used to collect (some of) the data
- Where are data used in the research project stored
- What data protection measures are used
- What forms of physical data are used in the research project
- What documentation is created and maintained to ensure proper the data can be properly interpreted
- How are the code scripts that are used for processing and analysing data managed
Is the data used in the research project collected from and/or about human beings?
↩︎
NB: Two “No” responses to this question were changed to “Yes” because
upon investigation of the data types collected, they were clearly from
human beings (e.g. BSNs)
What research populations are studied in the research project?**
↩︎Overall response
By department
What types of data2 are used in the research project?**
↩︎Types of data asked about in the survey included general data types3 & specific data types. During analysis a basic risk categorization level was applied to the data types reported.
General types of data**
Overall response
By department
Specific types of data**
Overall response
By department
Specific types of data classified into risk categories**
Overall response
By department
What software programs are used to collect (some of) the data?**
↩︎Overall response
By department
Where are data used in the research project stored?
↩︎All data in one location
Overall response
By department
With data protection measures
Data in various locations**
Storage locations listed in the survey
Storage locations listed in the survey plus data protection measures
Other storage locations reported by respondent
Data risk categories in various locations**
Overall response
By department
By research role
Other storage locations - overall response
What data protection measures are used?**
↩︎Overall response
By department
What forms of physical data are used in the research project?**
↩︎Overall response
By department
How are the physical data stored and protected?**
Destruction or preservation of physical data
Paper & physical audiovisual data
Are the paper & physical audiovisual data digitized?
Overall response
By department
What happens after the paper & physical audiovisual data are digitized?
Overall response
By department
What documentation is created and maintained to ensure proper the data can be properly interpreted?**
↩︎Overall response
By department
Documentation standards5
Overall response
By department
How are the code scripts that are used for processing and analysing data managed?**
↩︎Overall response
By department
Conclusions/Recommendations on data & research populations
↩︎- Almost all of the data studied at FGB comes from human beings either directly or indirectly (i.e. data reuse). This means the GDPR needs to be in the back of our minds for basically all FGB research; this does not mean the GDPR always applies, just that all FGB researchers need to be aware that there’s a high likelihood it could apply to their research.
- 7% of FGB researchers are collecting BSNs or other national ID numbers, with the highest rates in KNOP. It is imperative that researchers understand that we are not allowed to collect BSNs for research purposes, even reimbursement purposes.
- Very few alternative responses were provided by respondents for
types of research subjects or data, meaning the faculty has a good sense
of what populations and data types are being used regularly in the
faculty
- This information will be used as a starting point to define best practices and protocols for these regularly studies data/populations to avoid researchers needing to reinvent the wheel with every research project
- Most researchers store their data in multiple locations, and there
appears to be fairly good usage of the VU standard storage options,
Research Drive and SciStor.
- A fair amount of data is still stored on computer hard drives and/or external hard drives/USB sticks; it is therefore imperative that researchers keep these devices safe and clean up the data from these drives when it is no longer needed
- A fair amount of data is stored in “other” options, but unfortunately no further information was provided on what most of these other options are. We should continue to encourage and promote the use of the three standard FGB storage options: YODA, SciStor and Research Drive, as well as Teams in some rare cases.
- Some undesirable options for storage, such as DropBox, are still used, but fortunately this is fairly infrequent.
- The vast majority of physical data produced in the faculty is paper
data
- Most respondents are aware of the need to maintain the original physical data for the same duration as any digital copies, which is VU policy.
- Within KNOP, more than half of respondents who digitize their paper materials go on to destroy them, which is generally undesirable under VU policy (and, where applicable, WMO regulations). KNOP should ensure that their researchers are fully aware of when it is and isn’t appropriate to destroy paper materials after digitization. This is also a point of concern, to a lesser extend for ETP.
- Most data documentation takes the form of the code scripts used to
process and analyse the data. Fortunately, very few researchers generate
no documentation, but less than 40% create Readme files even though this
is a requirement for all data that are used for research publications
per the faculty archiving guidelines
- About 10% of respondents from BW report not producing any documentation; this is a point of concern that should be addressed in the department to ensure that BW data is understandable to its own researchers and to others
- Data documentation standards remain a complicated and unclear topic for researchers; REPS is responsible for clarifying this issue
- About 10% of respondents from KNOP and POW report no management of their code scripts, not even writing comments in their code. This is a practice they can improve by simply ensuring that the scripts or syntaxes they write include some basic comments of the process the researcher used.
Archiving Practices
Questions asked:
- Will the data used in the research project be archived
- When will data used in the research project be archived
- How long will data used in the research project be archived
- Where are the data used in the research project archived
- How is archiving of physical data handled
- How are third parties made aware of the archived data used in the research project
Will the data used in the research project be archived?
↩︎Overall response
By department
By research role (researchers only)
If data won’t be archived, what are the reasons for not archiving?**6
When will data used in the research project be archived?**
↩︎Overall response
By department
How long will data used in the research project be archived?
↩︎Overall response
By department
Where are the data used in the research project archived?
↩︎All data in one archive
Overall response
By department
Data in several different archives**
Archives listed in the survey
Other archives reported by respondent
Data risk categories in various archives**
Overall response
By department
By research role
Other archives locations - overall response
Are physical forms of data archived?
↩︎Overall response
By department
Where are physical forms of data archived?**
How are third parties made aware of the archived data used in the research project?**
↩︎Overall response
By department
Conclusions/recommendations on archiving practices
↩︎- Plans to archive data have greatly improved since the previous
assessment
- KNOP & POW should investigate their higher rates of not archiving data
- BioPsy may have reported lower rates of archiving due to confusion of roles & responsibilities for researchers regarding archiving
- Archiving rates appear much lower amongst assistant professors compared to other researcher roles; this may need to be addressed
- Most common reasons for not archiving were not knowing how to or the
reuse of existing data
- Role for REPS to improve understanding of practices and clarity on how to handle reused data and archiving
- Good numbers for archiving once research is complete, however rates of archiving as soon as a research article is published should be improved
- When data are all archived in one location, YODA appears to be the
primary archive used, which is the preferred solution for FGB
- 6% of respondents reported archiving all data on a USB or external hard drive, which while small is still a major concern, particularly for ETP and BW
- When data are archived in several locations, “other” archives were
the primary location, but respondents provided no indication as to what
these “other” archives are.
- YODA uptake when data are archived in multiple locations is much poorer, and unfortunately far to much data in these cases is “archived” on computer hard drives or USB/external hard drives
- The faculty needs to continue to improve awareness of YODA as the preferred archive for almost all FGB data, with OSF as an additional option for archiving documentation
- Archiving of physical data appears to be fairly high, however ETP
should investigate if physical data archiving rates can be improved
- Physcial archiving locations remain unclear to researchers; REPS should continue to improve awareness
- Rates of archived data registration in PURE are fair (~25%), but
need to be much higher as PURE registration after archiving is required
for all archived datasets; REPS will continue with awarenss on
this topic. BW should particularly look to improve these practices
- Most commonly researchers cite providing information in the research article, but the VU requires PURE registration
- ~10% of respondents (~20% of BW) do not do anything to make third parties aware of archived data; this needs to be improved, particularly within BW, as these practices are the bare minimum for making data findable
Data Reuse and Research Publishing Practices
Questions asked:
- Will the data used in the research project be available for others to reuse
- Has consent been obtained for reuse of data used in the research project, where applicable
- How are third parties made aware of restricted access data
- For which purposes may the data used in the research project be reused
- What plans are in place to handle requests to access restricted access data
- What hurdles have been faced with open access publishing of research articles
Will the data used in the research project be available for others to reuse?
↩︎Overall response
By department
By data risk category
If data won’t be available for reuse, why not?**7
Has consent been obtained for reuse of data used in the research project, where applicable?
↩︎Overall response
By department
By data risk category
How are third parties made aware of restricted access data?**
↩︎Overall response
By department
For which purposes may the data used in the research project be reused?**
↩︎Overall response
By department
By data access level
How will others be informed of these reuse purposes?**
Overall response
By department
By data access level
Is there a plan for handling requests to access restricted access data?
↩︎Overall response
By department
Who will handle requests to access restricted access data?**
Overall response
By department
Has the plan for handling access requests been documented somewhere?**
Overall response
By department
What hurdles have been faced with open access publishing of research articles?**
↩︎Overall response
By department
Conclusions/recommendations on data reuse and research publishing practices
↩︎- FGB researchers clearly want to make data available for reuse, but
many (~60%) plan to provide access to (some) data upon request
- Plans to share data have improved greatly since 2019 when only ~50% of research planned to share their data
- Support for the handling of restricted access data is particularly important for KNOP & POW
- ETP overwhelmingly openly publishes data, but investigations are needed to see if this is being done in a GDPR-friendly manner
- Many researchers are openly publishing data that could be higher
risk and many more are not openly publishing lower risk data
- Additional support on understanding data risks and when data should and shouldn’t be shared openly is an absolute requirement going forward
- Lack of consent was the primary reason for not sharing data if it
won’t be shared
- Improving and clarifying consent procedures around data reuse are absolutely necessary
- A sizeable number of researchers (20% overall and 33% for BW)
believe consent is not necessary to reuse data because the data are
anonymous/anonymized
- Currently, consent practices do not appear to line up well with the risks posed by the data and there appears to be confusion on what consistutes anonymous/anonymized data
- Further investigation into this topic is necessary to determine how accurate that conclusion is, and how to better support researchers on this topic
- Similarly to the archiving practices, most researchers make others
aware of data that can be reused via information in a research article
- This is a reasonable option, but shouldn’t be the only option. Data are much less likely to be found for reuse if they are only mentioned in passing at the end of an article
- PURE registration of datasets is also necessary for the VU to monitor data archiving and data publishing practices, but only 23% of respondents register their datasets in PURE
- More than half of researchers reported that their data will only be
used for new research questions, but ~25% have no plans to inform
potential uses about this limitation
- The lack of documenting and reporting on reuse purposes is particularly high for BW at 35%
- ~30% of publicly available data will not include any information on reuse purposes, which prevents that data from being appropropriately reused, particularly if the data is only to be used for new research questions (>60% of publicly available data is intended to only be reused for new research questions)
- Almost 50% of researchers with restricted access data do not have a
plan for handling data access requests
- This is particularly concerning for KNOP and POW for which,
respectively, 60% and 67% of respondents do not have a plan
- These rates for KNOP and POW are about the same as in 2019; the reporting from other departments in 2019 may not be reliable due to small response rates, but at the time, 100% of respondents from the other three departments reported that there is a plan in place for handling data access requests.
- Over 20% of respondents with a plan have not documented the plan and >10% do not know if the plan is documented
- The most commonly cited party who will be responsible for handling data access requests is the PI and then first author.
- This is particularly concerning for KNOP and POW for which,
respectively, 60% and 67% of respondents do not have a plan
- Most researchers (>50%) report not facing any hurdles when
publishing their research articles as open access.
- The most commonly faced issue, particularly for BioPsy, is that the VU doesn’t have an agreement with the journal the researcher wishes to publish in.
Footnotes
Consideration was also given for respondents with multiple research projects. Researchers were advised to answer the questions by thinking about their main research project or the one that uses most of their time. If their time was equally split between projects, they were advised to answer the questions by thinking about the projec tiwh the most complex ethical, privacy and/or RDM concerns. Research supporters were asked to answer the questions based on their most common experiences if they work on multiple projects.↩︎
Questions that simply ask about data, refer to digital data, which was explained to respondents. Physical forms of data were addressed in separate questions.↩︎
This question was posed in order to narrow down the number of options shown in the next question on specific data types.↩︎
These results were not analysed by department because the overall number of respondents using biological data was already very small.↩︎
These results have limited reliability because most respondents who said they used standards for documentation did not appear to understand the question based on the answers given to the follow-up question on the standards used (i.e. the standards that were reported were not standards). The follow-up question on standards used was not analysed because the results were too disparate and unclear.↩︎
These results were not analysed by department because the overall number of respondents who reported that they would not archive their data was already very small.↩︎
These results were not analysed by department because the overall number of respondents who reported that their data would not be available for reuse was already very small.↩︎